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ABSTRACT 



This paper reviews the literature on transformational change 
in education and presents a case study of such change on a university campus 
on which the student evaluation of faculty (SEF) form was being revised. An 
enactment model of transformation, relying on the work of J. Everett (1994) 
is used to interpret the main events of the case analysis, and the specific 
strategic decisions made by the change committee are summarized in the form 
of a planning process for use by others interested in revising SEF forms. The 
SEF previously used was summative in nature in that the results for each 
instructor were compared to the means across the department, college, and 
university. The approach taken presumes that SEF forms, when 
transformational, are dynamic and themselves undergo frequent change. The 
validity and reliability of the new SEF, which did not permit the use of 
numerical averages or rankings for summative purposes, were studied in pilot 
studies and after a year of use. Item level data from actual student 
responses (22,518 responses in 1998 and 21,265 responses in 1999) are 
interpreted as providing initial support for the transformational effects. 
Some preliminary data are also provided on other transformations of teaching 
and related activities at the university. The new SEF is attached. (Contains 
3 figures, 4 tables, and 28 references.) (SLD) 
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This paper describes a process by which teaching and learning is being transformed on a 
university campus. Most universities evaluate teaching using a student evaluation form (SEF) and 
from time to time, committees are charged to revise them. Revising such a form is an example of 
routine system change that can provide an opportunity for transforming definitions of teaching and 
learning, but which often becomes mired in politics or trapped in bureaucratic inertia. At the time of 
the revision in our case, the internal and external conditions were ideal for transformational change. 
The committee recognized the opportunity created by those conditions and chose strategies that 
fostered transformational changes in the teaching and learning process. This paper reviews the 
literature on transformational change, especially as it applies to structural inertia, and provides a case 
example. An enactment model of transformation is presented ( Weick, 1 979, Everett, 1 994) and used 
to interpret the main events of the case analysis. The specific strategic decisions made by the 
committee are also summarized in the form of a planning process for ease of use by others interested 
in revising SEF forms. The approach taken here presumes that SEF forms, when transformational, 
are dynamic and undergo frequent change. The results section reports the validity and reliability of 
the new SEF at the end of the first year and item level data are interpreted as providing initial support 
for the transformational effects. In addition, preliminary data are provided on other transformations 
of teaching and related activities. 

TRANSFORMATIONAL DYNAMICS IN ORGANIZATIONS 

The literature describes a "transformed" organization as one in which all members are 
empowered, able, and motivated to experience self-initiated responsibilities regarding their roles 
(Bryman, 1996). Transformations are accomplished through communication among organizational 



members - i.e. that is, transformations are driven by talk and action (Everett, 1994). In Education, 
transformation generally refers to outcomes such as empowered learning by students (Marshall, 
1 995). Such outcomes are argued to require communication changes in the system in which learning 
is framed and enacted (Boyd & Myers, 1988). In order for true educational transformation to occur, 
"Shared decision making and shared leadership from multiple perspectives is essential" (Alexander & 
Keller, 1994). 

Even though transformational goals assume decentralization and empowerment, the literature 
on transformation tends to argue or assume that some top-down process will be necessary to 
transform a traditional organization (Bryman, 1996; Marshall, 1995). The implication is that 
bureaucracies create powerful forces for inertia. Such forces overwhelm new ideas in one part of the 
system and thus, the new ideas do not get carried through the organization. Hence, powerful 
leadership is seen as necessary in traditional organizations because it creates vision for inspiring 
change and authorization for disrupting inertia. 

The case discussed in this paper represents a common situation in which a committee of 
individuals in a bureaucracy is empowered to induce change. The committee's task is embedded in 
complex bureaucracy (in our case, the form to be revised was written in the union contract, used by 
administrators for faculty evaluations, and had an existing administrative infrastructure). To succeed, 
the committee's recommendations for change had to compete with alternative ideas and survive inertia 
associated with each part of the bureaucracy affected by the change. Typically, such committees are 
charged with responsibilities for change without the benefit of executive level vision. To inform 
transformation strategies under such typical conditions, a decentralized model of leadership activities 
is required (Bryman, 1996). 
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Wheatley and Kelner-Rogers (1996), Everett (1994), and Weick (1979) provide guidelines for 
such a model. They assume that members' activities create the specific characteristics of the 
organization. The continuous actions and communications of members are the mechanism by which 
organizations are maintained and changed. In this sense, organizations are "events" that are enacted 
by their members. All actions by members create changes that are potential transformations. Most of 
the actions are not repeated or adopted by others, and thus, they do not lead to transformations. 
When they are enacted, however, their consequences become components of the organizatioa In this 
sense, enactment is the mechanism by which organizations are changed. The next section presents a 
model that describes the transformation process as a sequence of enacted organizational changes. 

An Enactment Model of Transformation 

Figure One summarizes an enactment model of transformation. The model relies heavily on 
the work of Everett ( 1 994). It assumes at any given point in time, an organization is a complex set of 
enacted events and meanings. According to the model in Figure One, the particular conditions ofthe 
current external and internal environment place demands on members for action. As individuals 
respond to environmental demands, they use their own history to change at least the internal 
environment to some extent. This created change is a new environment of the moment. Since 
individuals experience different demands and bring different histories to the situation, there will 
always be a variety of responses to any current set of conditions. The range of responses creates a 
variety of enacted environments, each of which is then a potential transformation state. 

In organizations, the individual actions do not occur in isolation. Instead, they demand 
interactive responses of more than one actor. As individuals continue to interact, some environments 



are reenacted and others are not. A response to any enacted environment for any reason at all selects 
it for the organization. Actions always introduce some change into the selected environment, hence 
the environment is dynamic. When an environment is regularly selected and reenacted, it is retained. 
The retained environment is behaviorally incorporated into existing patterns of actions, i.e. the 
organization has reached a transformed state. 

The new transformation state has outcomes that leave the organization in a new relationship 
with the environment. Transformed states constitute a set of new initial conditions for subsequent 
action. These conditions include new definitions of the organization, its components and its activities. 
Retained environments will be selected more often than novel environments. Thus the set of enacted 
environments is less variable than all the other possible potential environments might suggest because 
selection favors retained environments. This behavioral model, then, is also a model of adaptive 
change. 

Bureaucracies exercise leadership through the selection and retention among the many 
environments enacted by their members. The task any committee in a bureaucracy faces is to gain 
system wide acceptance of its proposals. According to our model, this requires the adoption of 
strategies that increase the likelihood that a committee's proposed changes are in fact selected and 
retained by others. In particular, the goal is to have the proposals incorporated into administrative 
routines, thus mobilizing structural inertia in their favor. 

A general version of the model described above helped the committee make strategic 
decisions about how to proceed and is used in the following sections of the paper as a heuristic to 
interpret and analyze the committee’s process and its outcomes to date. 
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THE CASE 



Youngstown State University (YSU) is a metropolitan campus serving 12,500 students, with a 
unionized faculty. Its original SEF was developed in 1974 and remained unchanged until 1993. The 
specific questions on the SEF were written into the contract with the faculty union. As such, the form 
represented issues that were contractual conditions of employment and implied a single pedagogical 
approach to instruction (i.e. a textbook, lectures and exams). For example, six of the twenty-six 
questions were of a contractural nature (e.g., whether the professor kept scheduled office hours and 
was on time for class). Seven of the twenty-six questions dealt with pedagogical issues, two 
specifically about exams in the class and four specifically about the presentation of the material. No 
allowance was made for the possibility that the professor might have used other instructional or 
assessment methodologies. 

This form was summative, as opposed to developmental or formative, in that the results of 
every question for each instructor were compared to means across the instructor’s department, 
college, and university (Scriven, 1981). These comparisons were used to inform tenure and 
promotion decisions. One specific question asked the student if “Overall, this instructor is among the 
best teachers I have taken”. The responses to this question and the average response to 24 of the 
other questions were used in a summative, rather than developmental way. At the time, these were 
the only assessment tools used on campus to evaluate faculty teaching. In addition, there were no 
university-wide teaching development programs. 

The summative nature of the form and its status as a negotiated contract item thus defined 
teaching in an adversarial way, inhibited change, and helped to suppress teaching development 
activities. The form also functioned as a set of normative constraints on teaching pedagogy, due to its 
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emphasis on the traditional lecture model and objective testing. These characteristics implied a 
definition of good teaching in terms of traditional pedagogy and compliance to contractual matters. 

A Change in Initial Conditions 

Eventually the definition of teaching reflected in the SEF foiled the needs of the campus. 
Changing external accrediting expectations and the introduction of new and diverse educational 
philosophies led to union and management consensus regarding the need for change in the SEF. As a 
result, a Student Evaluation Review Committee (SERC) was appointed to develop a new student 
evaluation of teaching form for classroom use. The form developed by the SERC (hereafter referred 
to as the interim SEF) was written into the labor contract. SERC developed a form that did not 
permit the use of numerical averages or rankings for summative purposes. 

The interim form had 15 questions, six of which asked detailed demographic information 
about the student which made it possible for faculty to identify students. Another three questions 
asked questions specific to the compliance with the union contract, such as whether the student 
received a syllabus and whether the faculty member adhered to office hours. One of the questions of 
a contractural nature had multiple responses including whether the professor often missed class, 
started and stopped class on time, dismissed class early on a regular basis, and used class time 
efficiently. One question specifically asked about the room, whether it was too cold or hot, poorly 
equipped, too crowded etc. The topic of one question was academic dishonesty. Two other 
questions dealt with whether the professor returned work in a timely fashion and whether the 
objectives of the course were met. Only two questions specifically dealt with classroom pedagogy. 
One of the two pedagogical questions asked if the course helped the student improve in eleven 



distinct skill areas ranging from a theoretical understanding of the subject to computer skills. 
Students were able to choose as many responses as they felt were appropriate. This question 
introduced new ways in which a student might learn, but the multiple response categories made 
interpretation and analysis difficult, both developmental^ and summatively. The other pedagogical 
item dealt with student experience and asked if the course helped the student to reach six specific 
goals r anging from “better understand myself’ to “respect the opinions of others” to “appreciate the 
importance of this subject”. Again, the multiple response nature of this item made it difficult for 
faculty to use the results for development and made it impossible for administrators to use the results 
for evaluation. 

This interim form represented an attempt to dramatically shift past goals of the SEF. 
However, the use of multiple response items and the number of items that did not deal with the 
teaching and learning process led to general dissatisfaction with the form among the campus 
community. Internal dissatisfaction led to the appointment of a committee on teaching development, 
teaching environment and teaching evaluation (CTD). The CTD was immediately charged with the 
task of revising the interim SEF. 

An Opportunity for T ransformation 

This newly developed committee, the CTD, recognized that the current campus culture and 
accreditation demands created a set of conditions that made it possible for the outcomes of the SEF 
revision process to be transformational, i.e. remove teaching from an adversarial context and 
transform the definition of good teaching and learning. Revising the form presented an opportunity to 
engage the campus in a discussion of what good teaching should mean in the future. Such a 



redefinition could affect not only classroom pedagogy, but also such matters as the conditions of 
employment, the scope of the union contract, and ways in which administrative summative needs are 
met. 

T ransformational Strategies 

To produce a stable state with transformed meanings for teaching, it would be necessary 
for a great number of people on campus to enact proposals of the CTD. The Committee used this 
insi ght to inform its decisions summarized on Figure Two, which lists a set of four general 
decisions to be made in any revision of a SEF. Decisions about that list (whether deliberate or 
not) form general norms that guide the SEF revision process. The committee’s decisions were as 
follows. 

1. Strategic Approach for Ensuring Selection and Retention. The committee decided 
that in order to achieve acceptance of a revised form, and to maximize transformational 
consequences for teaching and learning, the revisions would have to be grounded in literature on 
good teaching. The way in which the committee spoke about teaching could then be couched in 
substantive as opposed to political terms. It was the committee's intent to avoid individual and 
group interests and positions (i.e., sources of inertia and opposition) in the initial framing of the 
revision. Thus, the committee engaged in a thorough examination of the literature on teaching 
evaluation. 

2. Strategic Choices of the Goals of the Form. The literature addresses the relative 
merits of several commercial student evaluation of teaching forms (i.e., Center for Faculty 
Evaluation and Development, 1994; Educational Testing Service, 1992; Marsh & Roche, 1997; 



Tagamori & Bishop, 1990). Commercial forms allow for national comparisons based on 
established norms. However, the committee believed that an SEF form self-designed by our 
faculty would have the benefit of behaviorally implicating faculty in the transformational process. 
Self-designed forms tend to force a discussion of the definition of good teaching and allow 
altering that meaning over time. These benefits were consistent with the Model’s criteria for 
selection and retention. Thus, the model provided goals that enabled the committee to evaluate 
literature based alternatives with regard to the YSU environmemt. 

There is also a debate in the literature regarding the value of student evaluation forms 
(Tuckman, 1995; Herbert, 1995; Broder & Dorfinan, 1994; Hampston, Viehmeyer, et. al., 1998). A 
major issue in this debate is that the goals of SEFs are often multiple and unclear (e.g., summative vs. 
developmental). Additionally, even when grounded in sound theory, some authors argue that the 
results from student evaluation forms may suffer from student biasing factors and problems as a 
function of the partisan interests of those who use the form (Carey, 1993; Cashin, 1988; Greenwald & 
Gillmore, 1 997; Herbert, 1 995; McKeachie, 1 997; Seldin, 1 993). However, others argue that there 
is little evidence of student bias and call for the continued use of SEF measures (e.g., d’Apollonia & 
Abrami, 1997; Marsh & Roche, 1997). Despite the ongoing controversy over the use of SEF 
measures, they continue to be used in higher educational settings in general and at YSU, specifically. 

Thus, the committee decided to maintain the focus on formative goals even at the expense of 
the form’s summative uses and even if the form was not free of all bias. This decision was 
collaborative in that it reflected respect for institutional heritage by integrating into the committee’s 
work, the history and commitment from the SERC, the committee that created the interim form. 
Additionally, this decision served to implicate others (including past members of the SERC) 



behaviorally in the revision activities. Thus, the committee agreed to frame its activities as a forward 
development of prior activities regarding the form, rather than starting “anew.” The Committee's 
choices reflect the strategic goals for inducing selection and retention described earlier in the Model 
presented in Figure One. As a result of these goals, the Committee chose a self-designed SEF form. 

Finally, the committee analyzed particular intellectual frames from the literature on 
dimensions of good teaching, selecting Cashin (1989) for its simplicity, face validity, and 
consistency with committee goals. Anchoring its discourse in that definition helped ensure that 
forward development was not cast in the political terms of the past. It also grounded discussion 
in one set of intellectual language. This process makes self-interest language by others apparent, 
and forces others to use a new language to discuss the form and its process. 

3. Collaborative Strategy for the Revision Process. To secure the system- wide selection 
and retention of its proposals, the committee sought to develop a revision process that was as 
collaborative as possible, thus implicating the activities of as many members of the campus 
community as possible. The composition of the CTD also reflected a commitment to 
collaboration among multiple constituencies on campus and defined the task as a campus- wide 
undertaking. The committee represented various sectors of the campus, including chairs, deans, 
faculty, and administrators. The union appointed three faculty members to the committee and the 
adminis tration appointed three administrators: the Dean of Education and two department chairs 
representing two additional colleges. The appointed faculty were both senior and untenured 
junior faculty. The committee's composition ensured expertise in assessment, educational theory 
and evaluation, experience with the form's different uses (e.g. chairs, promotion committee 
members, faculty etc.), statistical analysis, knowledge of the relevant campus cultures and the 



commitment to a theoretically-based revision of the form. 

The committee developed multiple methods for enhancing the collaborative nature of its 
process by seeking input from all campus groups. The committee solicited the concerns of 
students, faculty and administrators about the old SEF Form and the Interim Form. Every faculty 
member was invited to submit examples of questions they developed and used to assess his or her 
teaching. The CTD also met with the deans and chairs regarding their needs and experiences with 
the previous forms, and it met with individuals on the administrative/union negotiating team. 

Later in the process, the committee sent out reports to all these parties on its activities (including 
student government), and it conducted several voluntary pretests on the proposed revised form. 
The pretests included a form asking volunteer faculty for their reaction to the form and the full 
implementation of the new form included a survey inviting student reaction to the form. The 
objective of the CTD was to involve all interested parties in decisions regarding the revision in 
order to increase their behavioral and communication activities about the newly revised SEF 
(Centra, 1993). 

4. Sources of Inertia and Opposition : The CTD believed a collaborative process would help to 
"identify" the relevant sources of inertia and opposition as well as engage those sources in 
communication and other behaviors about the new form and its related processes. In particular, the 
committee formally interacted with others about the following: (a) the Union’s concerns for job 
protection, (b) deans’ and chairs’ concerns for summative evaluation of teaching, (c) faculty 
concerns regarding student bias, (d) staff concerns regarding the effort and cost involved in any 
change in the form or its administration, (e) concerns by students, faculty and administrators about 
instructional quality, (f) concerns by faculty who had developed the interim form, and (g) concerns 



of various programs about the form’s status in terms of their accreditation criteria. Members of these 
interest groups were contacted, urged to reply to the committee, and informed as to the CTD's 
decision regarding their specific concern. Whenever possible, the committee attempted to incorporate 
in its activities information outlining its attempts to address the concerns of the interest groups. The 
CTD wanted all groups on campus to feel connected to the form in some way. The committee felt 
that by including all possible sources of inertia and opposition in the form's composition, the form was 
more likely to have a transforming effect on teaching. 

Since the initial conditions included mandates for change, the Committee was not required to 
sell the need for such change. It justified its activities by assigning the requirement for change to 
mandates from others. This further implicated the actions of others in the Committee’s 
undertaking. It also tended to dissociate negative feelings about the need for change from the 
committee itself and from its specific activities. 

Enacting the Revision Process 

In keeping with the goal to honor heritage and past committee work, the CTD began by 
categorizing each item on the original and interim SEF using the conceptual dimensions of teaching 
effectiveness in Cashin (1989). As a result, the committee learned which categories were and were 
not addressed by the interim form, and identified items that did not address teaching effectiveness at 
all. This generated a list of questions on the interim form that needed to be revised. For example, the 
response categories in the two pedagogy questions on the interim form needed to be considered each 
as an individual item. The detailed responses in the contractural items needed to be clarified as 



separate items. 



The committee also listed concepts that were under represented or not addressed by the form 
at all. The seven dimensions from Cashin (1989) were collapsed to the following four categories: 
course design, availability to students, administrative requirements, and assessment of 
instruction/subject mastery/delivery of instruction. Figure Three lists the four categories of the 
conceptual dimensions of teaching and provides a definition for each. These dimensions became the 
anchors of the committee's discussions about assessing teaching and brought the legitimacy of the 
literature to the discussion. The conceptual dimensions were inclusive and non-political in content, 
yet invoked contributions of the various constituencies. 

From all the information, the CTD created a revised form that was introduced in a year-end 
report to the Union, negotiating team, Provost, Deans, and others. Messages associated with earlier 
solicitations for information prepared the various constituencies for much of what they found in the 
report. However, for selection and retention to occur, items in the report must themselves be 
behaviorally addressed by others (see Figure One). To that end the committee scheduled feedback 
meetings that forced such activity. When others participated in the feedback meetings, they 
contributed to the transformational adaptation process, even when critical. Criticisms are arguments 
for change and participate in the selection and retention of the recommendation because they treat the 
recommendation as a potential future state of the organization. The Committee’s collaborative 
activities, then, were a transformational leadership process involving members of the full range of 
constituents. 

The early collaborative strategies of the Committee activated the main political concerns about 
the form and its use. The early strategies also helped to identify weaknesses in the existing forms in 
terms of meeting the needs of various constituencies. Repeated collaborations with the various 



constituencies continuously involved them in the enactment of the new SEF. By using multiple 
methodologies (e.g. surveys, meetings, reports), the committee was able to access a range of 
reactions, concerns, values and interests. Toward the end of the process, the committee was able to 
construct surveys that led to very specific suggestions for change with a minimum amount of political 
negativity and rancor. Based on the surveys, suggestions were presented in a helpful and cooperative 
tone, and many of the suggestions were used in the final revision. In addition, these activities 
represent the multiple selection and retention process of the enactment model. These same 
interactions served to define earlier SEF forms as no longer satisfactory, thus weakening the inertia 
associated with those forms specifically. 

The Committee exposed the campus community to details of the new changed form in several 
pre-tests of early drafts, the results of which were shared with both the union and administration. The 
behavioral demands associated with actually administering the pretests reduced inertia and prepared 
the campus for final adoption of the form. This use of the new form was a strong force for 
widespread adoption of the form. In the absence of an alternative, the revised form would likely be 
selected and retained since the union contract required evaluation using an SEF. The older forms had 
been discredited and their routines had been replaced by the new ones for several quarters due to the 
pre-tests. This shifted the inertial forces to the older forms. Thus, the CTD exercised leadership in 
the selection and retention process by forcing collaborative involvement, systematically articulating 
problems with the established forms, and piloting the new form. Although repeated requests for 
feedback were not always successful, overall, the activities that were demanded by the committee 
were critical to the success of its revision process. 

Table A provides a summary of the specific steps the committee took as it revised the SEF. It 
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also summarizes the initial conditions faced by the Committee as it undertook that task, its strategic 
decisions and the timeline of the process. The information is presented in Table A as a planning 
process so that the events can be seen narratively, rather than organized around the Model. This 
provides readers who are facing similar tasks a practical format within which to see the steps in the 
process without any connection to the conceptual details of the enactment process. 

METHODS 

Based on the year-end report, the committee had come up with a first draft revised student 
evaluation of teaching form. The form was modified over the summer and then pre-tested in the 
Fall of 1996. This first pilot involved 37 classes and 633 students. Based on the results of this 
pilot, the form was again modified. The modified form was then piloted in the Spring of 1997. 

The second pilot included data from 136 classes and 2763 students. In each of these pilots, 
participating faculty were surveyed. Based on the results of the second pilot and the comments 
from students and faculty, a few minor changes were made to the form. The form developed as a 
result of the two pilots was presented to the negotiating team and accepted during the Summer 
1 997. Finally, in the winter 1 998, the new SEF was used in its first campus wide adoption (see 
Appendix for a full list of the current questions). 

The CTD agreed to continually review and update the form. Thus, in the Winter 1998 and 
Winter 2000, faculty were surveyed as to their reactions to the form. See Table B for the four 
basic questions asked of faculty in the 1998 survey. There were 1 13 responses from 
approximately 700 surveys distributed. The 2000 faculty survey added two additional questions 
(See Table B). Out of approximately 700 surveys distributed, there were 52 responses. 



Finally, the CTD conducted a survey of students after the Winter 1998 evaluation period. 
The committee placed a copy of the SEF and a survey in the student newspaper. There were 
drop-off boxes in almost every building on campus. The 1998 student survey questions are 
identified in Table B as well. Only 47 responses were received. Finally in 1998, the committee 
surveyed the chairs and deans on the new form and 22 responses of 43 were received. 

The new SEF has now been used for three years. Each Winter quarter, by union contract, 
all faculty teaching that quarter are evaluated with the SEF. This paper also uses data generated 
from actual student responses to the SEF items for both 1998 and 1999. The 1998 SEF data 
(22,518 student responses) and the 1999 SEF data (21,265 student responses) are used in the 
results section. The validity and reliability of the form for Cashin's model are evaluated using the 
1998 SEF data, along with the results of the two pilot studies. Then, the 1998 and 1999 SEF 
data are analyzed to help determine if student's attitudes and expectations of teaching have 
changed since the implementation of the form. In addition, the results rely on information from 
surveys conducted by the CTD. The results of the surveys are reported in the next section and 
helped the committee make statements about acceptance of the form and the form's role 
transforming teaching. 



RESULTS 

There is no longer any discussion of returning to older forms, although there is discussion, led 
by the Committee, about continuing to improve the current form. Although we have no reason to 
think that the form is loved by everyone, it is in use, and accepted as legitimate. The selection and 
retention of the SEF occurred without undue frustrations, hostile expressions of interests, or negative 
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confrontation among constituents. 

The results associated with the adoption of the SEF are reported in two sections. The first 
addresses validity and reliability issues related to the SEF as a psychometric instrument. Data from 
the second pilot investigation and from the 1998 SEF data were analyzed to determine its reliability 
and underlying factor structure. The 1998 SEF data were tested to see how well they fit the factor 
structure identified from the second pilot. Conclusions regarding the instrument's relationship to the 
conceptual dimensions of teaching (Cashin, 1989) are provided. 

The second section investigates the transformational outcomes as suggested by the enactment 
model presented in Figure One through the use of the faculty and student surveys and by looking at 
responses to some of the individual SEF items. The outcomes are transformational states of the 
system not all of which can yet be evaluated. However, we provide preliminary information about 
those transformations that appear to be underway. 

Reliability and Validity of the SEF Instrument 

The literature used to develop the SEF was chosen for pragmatic reasons related to the 
collaborative nature of the process and for its face validity. The SEF items were intended 
primarily as dynamic indicators of transformational goals. The authors expect the items to change 
over time as understandings of good teaching develop. The authors also expect performance on 
new criteria (not addressed in any earlier SEF) to improve over time. 

Hampston and Rogers (1998) conducted the initial analyses on both sets of pilot data. 

The first analysis involved conducting an exploratory factor analysis of the data from the first 
pilot. For this analysis, the authors utilized a principal factors, factor analysis with an oblique 



rotation. Three factors were retained based upon a scree plot and eigenvalue analysis. Using a 
value of .30, these three factors were interpreted as Good Teaching, Advanced Technology, and 
Support of Student Learning. Eigenvalues for the three retained factors were 7.52, 2.35, and .87, 
respectively. Overall, the three factor solution accounted for 97% of the common variance and 
38.37% of the total variance in the scores. The intercorrelations of the factors ranged from .15 to 
.54. 

Following their exploratory analyses, Hampston and Rogers (1998) utilized confirmatory 
factor analysis (CFA) to test the fit of their three factor model to the data from the second pilot 
study. CFA is a procedure that attempts to assess the fit of a hypothesized factor structure to a 
new set of data. These authors reported that the hypothesized three factor model approached, but 
did not meet, acceptable levels of fit to the data based on recommended goodness-of-fit measures 
and cutoff values (Rogers, Abbey-Hines, & Rando, 1997). 

Utilizing the 1998 SEF data, we investigated the reliability and validity of the SEF again 
using the factor structure identified by Hampston and Rogers (1998). In this analysis, we first 
assessed the reliability of the three factor scales using internal consistency procedures. The 
resulting internal consistency reliability estimates for the Good Teaching, Advanced Technology, 
and Support of Student Learning scales were .91, .76, and .79, respectively. These results 
indicated that the SEF performed reasonably well within our sample in terms of reliability. 

Next, we assessed the validity of the SEF in our sample utilizing CFA. Again, the CFA 
procedure involved using the factor structure identified by Hampston and Rogers (1998) as the 
hypothesized measurement model. However, since we are early in the development process of 
the SEF, the results of these analyses should be interpreted cautiously. Table C presents the 
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resulting goodness-of-fit indices of the CFA. As indicated in the table, the tested three-factor 
model provided a reasonably good fit to the 1 998 SEF data. These results were interpreted as 
providing preliminary evidence for the factorial validity of the SEF and, in conjunction with the 
reliability analyses, support the appropriateness of our subsequent item-level interpretation 
procedure used later in the second section of Results on transformation. 

These results also have substantive implications for the conceptual dimensions of teaching 
used in the development of the instrument. For example, one factor that emerged in our analyses, 
Advanced Technology (factor II), was not specifically included in Cashin’s (1989) original 
conceptual framework. Therefore, our results seem to suggest that students on this campus see 
the use of technology as a separate criterion from those that define good teaching. 

Further, our results suggest that students do not distinguish between course design 
criteria, administrative criteria, and criteria dealing with subject matter mastery and delivery of 
instruction. That is, they see all of these criteria under a generalized rubric that we call Good 
Teaching (factor I). However, students did distinguish between "good teaching criteria" and 
other items dealing with teacher activities or course design features perceived as Support of 
Student Learning (factor III) (e.g. providing useful and thoughtful comments on student work, 
providing opportunities for the students to apply the materials and information learned in the 
course, or including a variety of methods and approaches designed to clarify the material). This 
factor is broader in scope than the "availability to students" category in Figure Three. With that 
modification, the Cashin categories have collapsed into two in our sample, one of which is a 
generalized good teaching factor and the other focuses on characteristics that provide support for 



student learning. 



Since all our strategies assume emergent changes in pedagogy and definitions of good 
teaching, the form will require ongoing revision and assessment as a key process for monitoring 
that change. Nevertheless, the ongoing assessment of reliability and validity of assessment 
measures is a crucial component in this process. 

Transformational Outcomes 

This section provides preliminary results regarding the expected outcomes of the 
enactment model of transformation that constitute new system transformation states. It is difficult 
to assess the system adaptations because we cannot distinguish the effects of SEF-related 
activities from the effects of other concurrent activities on campus (e.g. new faculty and 
administrators, accreditation processes, and the revision of the General Education Program). In an 
effort to address this unavoidable complexity of causal factors, the results reported here stress 
connections to the enactment model of transformational outcomes in Figure One. Thus, for 
example, data on student outcomes is not yet available and is clearly subject to the consequences 
of other changes in the teaching process. Although we cannot strongly document direct effects of 
the SEF revision process on empowered learning of students nor disentangle those effects from 
other causal factors, we can identify changes that, on the basis of face validity, should affect 
learning outcomes (e.g. changes in teaching definitions, innovation, and student understandings of 
key dimensions of teaching); and we can connect particulars of these processes to features of the 
model. 

A case analysis is never, in the end, a strong test of a theoretical model, but our data do 
support the model’s particulars about the dynamics of the various changes we report below. The 



dynamic process set in motion by the SEF revision continues to evolve. Since assessment is 
ongoing, this data is preliminary and the dynamic process is such that as development evolves on 
campus, the form will continue to change. The results reported below are preliminary, intended to 
provide guidance for the continuation of this process and for others involved in similar 
undertakings. The overriding objective is to be qualitatively rich in the description of processes 
described by the model and the connections of its abstract claims to particular events in the case. 
To reach that objective, we use multiple methods including analysis of survey results, student 
responses to SEF items, and participant observation by committee members. See Cassell and 
Symon (1994) for more examples of the use of qualitative research. 

The measures of these results rely upon student responses to the SEF, several surveys, and 
organizational reports about new programs and activities. In general, the response rate to all 
surveys was very low. Earlier in the process, responses to the committee were frequent and often 
negative. The reduced response rate and the low rate of negative responses are interpreted as 
indications of a lack of discontent with the SEF. That is, the authors interpret this result as a 
finding consistent with the enactment model’s predicted outcome of reduced variability. Not all 
results are based on those surveys. The results are also based on committee observations and 
communications. 

As a result of the revised SEF, the main transformation is that YSU now assesses teaching 
using criteria relevant to several accreditation organizations. It uses a process and tool that has 
campus-wide involvement in its design and that enjoys campus-wide acceptance. Indeed, YSU 
exceeds some accreditation requirements in that regard. The form has also been changed from 
summative in nature to formative. The results of the 1998 and 2000 Faculty surveys were essentially 



the same. In both surveys, 66 percent of the faculty respondents reported the form would help to 
improve their teaching and only 36 percent believe the SEF will accurately evaluate teaching. Thus 
the form is perceived as valuable for formative purposes even though its summative uses are not yet 
agreed upon. The Committee continues to send messages to the campus reminding them that we are 
evaluating the form with a view towards continuous improvement. The following sections summarize 
additional results that address specific issues in the model. 

1. Institutional Acceptance and Innovation. Based on the 1998 faculty survey (see Table B 
for a list of questions), when asked what they liked of the form, 84.1 % of faculty had something 
positive to say about the form, while 9.7 % said they liked "nothing" or "not much" and 6.2 % gave 
no response. When asked what problems they had with the form, of the 1 13 responses in 1998, 34 
respondents (30%) said "nothing" and another 12 (10.6%) had no response. The remaining 67 
responses included a number of areas. 1 The committee was surprised that only eight faculty 
commented on the questions asking of the use of technology by both students and the faculty 
member. These questions were a large area of concern by faculty early on in the revision process. 

Based on the 2000 faculty survey, when asked what they liked of the form, 86.5% (45 of the 
52 respondents) had something positive to say, 1 faculty said "nothing", and 6/52 (1 1.5%) gave no 
response. When asked what problems they had with the form, of the 52 responses in 2000, 1 3 (25%) 
said nothing and another 2 (3.8 %) gave no response. Of the 37 respondents who had something 
negative to say about the form, ten respondents mentioned the concern that student bias plays and 
either wanted some indication of how the student might be biased (whether the course was required, 



1 Some of the issues raised in this item were the following (number of respondents in parentheses): students are not 
competent to evaluate (5); questions irrelevant to course (10); technical usage question(s) (8); narrative difficult (6); 
not enough emphasis on the importance of the form (4); problems with a particular questions (5). There were an 
additional 29 specific concerns unable to be summarized. 
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how much time they spend on the course, how often they come to class) or a requirement that the 
student must explain why they strongly disagree or agree with an item. Nine people specifically 
mentioned the technology questions as not being applicable to all courses and an additional one 
person mentioned overall that not all items might be applicable to all courses (without specifically 
mentioning the technology questions). Both of these concerns represent sharp increases from the 
1998 faculty survey results. No other issue or concern received more than three mentions and they 
were similar in substance to the Faculty 1 998 survey. We were not surprised about the increase in the 
number of comments pertaining to the technology questions. This increase might be an indication 
that students are coming to expect the use of technology in most classroom situations and when it is 
not used, faculty feel some pressure to do so. Both the 86.5% approval rate and the decrease in the 
percentage of those who said they disliked "nothing" are encouraging. In both years, many of the 
respondents commented on the length and ease in using the form as things they liked about the form. 

Analysis of the comments from the 1998 student survey revealed no significant criticism (See 
Table B for a list of questions). Of the 47 responses, 25 of the respondents found no difficulty with 
the form and did not have any suggestions for improvement. The remaining 22 survey respondents 
did not have any consistent suggestion or problem with the SEF form. A few students expressed 
their confusion over the question concerning plagiarism and thought the question was asking whether 
they generally understand the definition of plagiarism. Finally, all 47 students answered "yes" to 
survey question #1 regarding whether the printed and oral instructions for fillin g out the form were 
clear. Except for the plagiarism question, the students said they understood all the questions on the 
form. Many of the chairs and deans who responded to the survey of the SEF said that, coupled with 
other indicators, the SEF could help to accurately evaluate teaching. 
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Thus, in general, the form seems to have been accepted by faculty, administrators and 
students. The union also accepted the form and the union-management negotiating team adopted it 
without change, complaint, or request for clarification. The team was also positive in its feedback to 
the CTD. The form has now become less political in the sense that the document is considered 
dynamic and changes to the form do not require re-negotiation of the union contract as was true in 
the past. 

We are not claiming that the form has no problems and is loved by everyone. Many of the 
deans and chairs find it not as helpful as they would like for summative uses. Many faculty find 
only part of the form helpful for formative uses. They still see some questions on the form as not 
appropriate to their class. They sometimes make suggestions for change. The dissatisfaction is 
leading to other transformations beyond the form itself. For example, departments, programs and 
colleges are now asking for faculty development plans that address teaching as well as research 
for summative purposes. Tenure and promotion committees are now permitted to use materials 
other than items specifically mentioned in the contact under some circumstances. Faculty are 
being asked to develop teaching portfolios that document their teaching activities with samples of 
work by students (to assess learning outcomes), of assignments and their goals, copies of syllabi 
that should include goals of pedagogy and learning outcome objectives. Until this past year, the 
campus attempted to use a single form as a stand-in for an entire faculty development/assessment 
program. Now, the form is seen as insufficient for such ambitious purposes. It’s inadequacies are 
stimulating creative local responses throughout the campus. 

These innovations are not yet system-wide transformations, but most sectors of the 
campus are in the process of generating potential transformations. For example, administrators 



have forwarded to the Committee requests by faculty whose teaching situations normally exempt 
them from evaluation (e.g. team teaching situations, supervision of practicums). Such faculty 
now want a form that works for their situations. They want to be evaluated. The Committee is 
working to encourage each College to develop supplements to the SEF for such situations and to 
share their proposals with other colleges. 

There are yet other examples of the form’s being used to appeal for change. Complaints 
about the lack of technology for classroom use and/or support for such technology now regularly 
cite the evaluation form as grounds. At a recent meeting of the Diversity Task Force a 
recommendation was made that each syllabus contain a statement regarding how the course 
addresses the item about creating an inclusive climate. 

This array of newly enacted potential transformations is somewhat ad hoc and 
inconsistent. It is, however, evidence of decentralized empowered activity about the formative 
and summative dimensions of the teaching/leaming process. A process of selection and retention 
will reduce the range of transformational states for specific problems. 

Also undergoing transformation is the scope of the union contract. That document now 
permits information in addition to the results of the SEF to be used in tenure and promotion 
decisions. The SEF is not in the body of the contract, but is an appendix. The contract is not 
clear about the SEF but implies that it is to be changed in an ongoing way and that such changes 
need to be approved by the joint negotiating team. We expect that the contractual matters 
relevant to tenure, promotion and termination will undergo significant change in response to 
enactments across campus. The committee does not expect that the form will be at the center of 
those changes. 
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2. Effects of SEF on Teacher Development and Pedagogy. The SEF is now an integrated 
component of teacher development and the formal evaluation process supports empowered teaching 
and learning processes. Developmental programs are beginning to be funded and there is movement 
on a larger teaching development program that relies on using SEF results. The administration is 
interested in the aggregate results of the SEF to ascertain problem areas and provide focused 
assistance and training. According to the enactment model of transformation, the use of SEF results 
as a guide to design workshops is an instance of selecting and retaining definitions created by the 
SEF. 

Teaching pedagogy has become a significant discourse on campus in a number of ways. There 
have been many workshops on campus covering factors such as peer review of teaching, active 
learning, critical thinking, use of the World Wide Web in courses and distance learning. In addition, 
discussion groups on certain teaching pedagogy techniques have formed and there is a new masters 
teacher program in Arts and Sciences. In addition, our campus has joined the Carnegie Campus on 
the Scholarship of Teaching and Learning. Beginning this academic year, we have had informal lunch 
discussions on teaching techniques. The members on campus have also begun to discuss the 
scholarship of teaching and learning and how we can promote that among others on campus. These 
discussions have been open to all faculty in the university and are being organized by a Carnegie 
Scholar as well as a group of faculty who attended the American Association of Higher Education's 
forum in March 2000 on "Campus Conversations: The Scholarship of Teaching and Learning." 

The Provost’s office has convened a meeting of the Committee with several other new 
committees on campus whose activities are intended to address teaching development. The 
recommendations were to develop a center to provide increased support of teaching using active 
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pedagogies and/or new technologies; to increase opportunities for diversity education for teachers 
and to provide workshops or mentoring for faculty on how to prepare a teaching portfolio for 
summative uses. Thus, the SEF continues to be embedded in bureaucratic structures and programs. 

3. Re-definitions of Teaching. The basic skills goals included in the SEF are now required 
in course outlines in many departments and colleges across campus. In particular, there has been an 
increased commitment to writing across the curriculum and to incorporate technology into the general 
education writing courses. Critical thinking skills are now assessed for majors and general education 
requirements. 

The authors anticipated that the new items added to the SEF form were ones that students had 
not previously considered to be central to good teaching. Thus, we expect the percentage of students 
who say "not applicable" (NA) to the new items to be higher than the NA percentage for the 
questions that were "old" definitions of good teaching. The results of this analysis are reported in 
Table D. In 1 998, the average NA response rate on all new questions was 9.87%. The average NA 
response rate on all old questions was 2.83%. Of this 2.83%, questions #12 (if the professor kept his 
scheduled office hours) had a NA of 19%. Presumably, students feel that NA is the appropriate 
response if they do not attend office hours. For the analysis, item 12 was removed and the average 
NA response rate for all other old questions was .8125 %, compared to 9.87% for new questions. In 
addition, the questions asking about the use of technology (#6, 1 8 and 1 9) are all new. The average 
response rate in 1998 for these questions was 21.87 %. Interestingly enough, the items that were 
already in existence on the old form were treated as appropriate and applicable in the student 
responses on the winter 1 998 SEF at a much higher percent than those new items on the form. 

The authors also expect that the rate of "not applicable" responses to decline over time as 
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students begin to learn these criteria are part of good teaching. Comparing the 1998 results to the 
1 999 results, the average NA response rate for new questions fell from 9.87 % to 8.99 %. Turning to 
the technology questions specifically, the average NA rate decreased from 21.87% to 20.37% and 
declined for all three items. In addition, the strongly agree responses for each of the three technology 
questions increased from 1998 to 1999. For item 6, the strongly agree response rate increased from 
30.5% to 34.2%; for item 18 from 30.6% to 33.4% and for item 19 from 30% to 33.3%. The 
decrease in the percentage of students indicating that the new technology-related questions were not 
applicable to their course, in conjunction with the increase in the percentages of students indicating 
strong agreement with those same questions can tentatively be interpreted as reflecting the 
transforming influence of the SEF on technology related teaching pedagogy. 

4. Students Are Empowered Learners. Effects of the SEF on actual student learning cannot 
yet be assessed. However, for the first time, campus assessment activities are looking at student 
outcomes and trying to identify trends in student learning. For example, the use of technology by 
students independently in laboratories across campus has increased dramatically over the past two 
years. Furthermore, students entering intermediate courses requiring technology use have higher 
competency levels at entrance. 

Limitations 

The case analysis design and the pragmatic use of the model for essentially political goals 
limits the verificational strength of our results. All research intended to directly inform practical 
action in a specific time and place face limitations. Complex causal settings cannot be subjected 
to the standard cannons of Science under these conditions. However, our primary objective is to 



provide guidance about a common kind of bureaucratic change situation. Our efforts have been 
to be specific and detailed in how the model can be used to inform strategies and interpretation of 
real events in a deliberate change process. The value of our findings lie in the expectations and 
outcomes that are distinctive to the model. Thus, we have weak results about student learning, 
but very interesting results about the relationship of evaluation items to student perceptions of 
good teaching, for example. Further, the results imply that successful change sets in motion 
multiple reactions to those changes throughout the system. Such changes are interpreted here as 
inevitable and as possible transformations dependent upon future selection and retention 

Since our effort is to provide guidance, we have also included on Table A a summary of 
the revision process in the case as a chronological summary organized as a rational planning 
process. This avoids the issues of science and verification associated with the model, while at the 
same time listing some issues raised by the literature as key factors in the success of planned 
change. 



DISCUSSION 

The results of this case analysis should not be interpreted as support for a search for 
“definitive measures” of teaching performance. Such a search can encourage a fixed, unchanging 
approach to solutions and tools, and unconstructive debates about first principles. Together with the 
politics that surround any assessment activity, inflexibility and "in-principles" debates can generate 
negative consequences and foster bureaucratic inertia. 

The results of the factor analysis suggested that students appear to use a generalized model of 
teaching (not specific factors). However, this definition does not include all items even on our SEF 
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form. Some of the items are not seen as relevant and do not load as part of the factor analysis. The 
fact that students use an undifferentiated model does not imply that they put every single item in good 
teaching. The results warrant farther investigation. 

The results reported here suggest that it is possible to create and revise tools and related 
administrative activities that identify and foster continuously improving teaching and learning goals 
and that assist instructors and administrators to develop pedagogy and programs to reach their goals. 
The results are especially encouraging to committees charged with changing activities in highly 
bureaucratized systems. The collaborative processes adopted by the committee in the case reported 
here could be used by any committee attempting to introduce change provided that there are external 
and internal conditions supporting change. Transformation requires the committee to be self-aware 
with regard to initial conditions, involve constituencies from throughout the system, and recognize 
enactment consequences of program activities. 
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FIGURE ONE: ENACTMENT MODEL OF TRANSFORMATION 
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FIGURE THREE: CONCEPTUAL DIMENSIONS OF TEACHING 
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TABLE A: SUMMARY OF THE SEF REVISION ACTIVITIES 



Initial 

Institutional 
Conditions 
(Initial Steps) 


Strategic 

Decisions 


Item 

Development 


Revision of 
Items 


Implementation 


Follow-up and 
Assessment 


ACTION STEPS 


Explicit 


Choose Literature 


Literature 


Circulate the 


Recommend 


Analyze student 


consensus from 


Based Approach: 


based 


newly revised 


adoption to 


responses from 


both union and 


review literature 


evaluation of 


SEF to 


union, 


complete 


administration of 


to identify 


interim form 


students, 


negotiation 


campus-wide 


need for change 


intellectual 




faculty, chairs, 


team, faculty 


evaluation 




strategies 


Solicit and 


deans and the 


and 




Controversial 




review 


union and 


administration 


plan future 


initial revisions 


Choose self 


additional 


schedule 




revision 


by ad-hoc 


designed form 


questions 


feedback 


Campus Wide 


processes 


committee 




faculty 


meetings 


use of new form 






Revise Form for 


members use 






survey of 


Appointment of 


formative goals 


as supplements 


Pretest revised 




students, faculty 


joint long 




to the current 


SEF, survey 




and 


standing 


Adopt 


form 


representative 




administration 


committee to 


Collaborative 




sample of 




about their 


revise form 


Methodologies: 


Develop new 


faculty and 




experience with 




collect and revise 


items to 


students 




the form 


Suspension of 


SEFs used at other 


address 








union contract 


universities 


selected 


Survey 






to allow for 




conceptual 


Analysis and 






experimentation 


Maintain 


criteria 


Revisions 






of form 


continuity with the 












past 


Circulate copy 


Pilot Study of 






Demands by 




of revised SEF 


new revision of 






accreditation 


Build on previous 


and schedule 


SEF and survey 






bodies 


drafts 


feedback 


of students and 










meetings 


all faculty 












volunteers 










Draft a report 












of years 


Survey 










activities to 


Analysis and 










union and 


minor revisions 










administration 








Fall 1994-Fall 


Winter 1996 


Spring/ 


Fall 1996 and 


Fall 1997, 


Spring 98 - 


1995 




Summer 1996 


Spring 1997 


Winter 1998 


Winter 2000 
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TABLE B: SURVEY QUESTIONS 



Faculty Survey Questions 


Student Survey Questions 


1 . What do you like about the form? 


1 . Were the printed and oral instructions for 
filling out the form clear? If not, what did you 
have trouble understanding? 


2. What problems do you have with the form? 


2. Is it clear to you when questions are not 
applicable for your classes? If not, please 
specify what kind of trouble you had in 
assessing the applicability. 


3. Do you think the results of form help to 
improve your teaching? How? How not? 


3. Was there something you wanted to say 
about a class or a professor that was not 
addressed by the form? If so, what question 
topic has been omitted? 


4. Do you think the form accurately evaluates 
your teaching? How? How not? 


4. Are there questions you do not understand? 
If so, which ones? 


5.* Have any of the questions on the form led 
to revisions of your classroom practices? 


5. Do any questions on the form seem 
repetitive? If so, which ones? 


6 * Do you feel that the wording of any of the 
questions on the form implies that you are 
expected to change any of your classroom 
practices? How? How not? 





* These two questions were added to the 2000 Faculty Survey and did not appear in the 1 998 
Faculty Survey. 
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TABLE C: GOODNESS-OF-FIT INDICES 




Index 



,2 



Value 

12616.78 (df — 149) p < .001 



NFT 

PI b 



.92 

.92 

.81 



Note . Nonsignificant chi-square values indicate a good fit to the data (Hatcher, 1994). AGFI = 
adjusted goodness-of-fit, NFI = normed fit index (Bentler & Bonett, 1980), PI = parsimonious 
index (James, Mulaik & Brett, 1982). indicates indices with .90 as recommended cutoff value 
for inferring model fit (Bentler & Bonett, 1980). ^he recommended cutoff for interpretation of 
the parsimonious index is .80 (James et al., 1982). Inspection of the normalized residuals for this 
analysis suggested that fixture psychometric work on the SEF should focus on refinement at the 
item level to insure that items are unidimensional and are assigned to the correct factors. 
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TABLED: ANALYSIS OF ITEM RESPONSES 
FROM 1998 and 1999 SEF DATA 1 



YEAR 


Old Items 2 
NA response 


Old without item #12 
NA Response 


New Items 3 
NA Response 


Technology Only Items 4 
NA Response 


1998 


2.83% 


.813% 


9.87% 


21.87% 


1999 


2.61% 


.688% 


8.99% 


20.37% 



Note: In terms of the SEF (see Appendix), 

1 Item number 23 was deleted from the analysis because it does not have a compatible response 
category. The status of item 1 5 with respect to new and old could not be determined, and thus 
was not included in the analysis. 

2 Old Items: # 1, 2, 5, 7, 9, 10, 12, 13, and 14 

3 New Items: #3, 4, 6, 8, 11, 16, 17, 18, 19, 20, 21, and 22 

4 Technology Only Items: #6, 18, 19 
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APPENDIX: SEF 



YOUNGSTOWN STATE UNIVERSITY 
STUDENT EVALUATION OF TEACHING AND LEARNING 

INSTRUCTIONS: Please enter your answers to the following questions by filling in the corresponding 
circle on the scantron sheet. For each question, select only ONE answer. Your response to these 
questions will provide helpful information to your instructor. 



DESCRIBE YOUR INSTRUCTOR'S TEACHING PROCESS BY USING THE FOLLOWING CODES: 


a. Strongly agree 


b. Agree 


c. Disagree 


d. Strongly Disagree 


e. Not applicable 



The Instructor: 

1 . followed the syllabus without significant deviations. 

2. developed the course to be appropriately challenging. 

3. designed tests/papers/projects which were consistent with the objectives of the course. 

4. designed assignments/papers/examinations requiring creative and original thinking. 

5. presented clear grading standards for this course. 

6. used advanced technologies to teach this course (e.g., multi-media, computers, Internet/World Wide Web, specialized 
lab equipment). 

7. appeared to have an extensive knowledge about the subject matter of this course. 

8. provided useful and thoughtful comments on papers/assignments/examinations. 

9. was open to questions and differences of opinion. 

10. was prepared and presented material in an organized manner. 

1 1 . created an inclusive classroom that communicated value for individuals and their differences (e.g. race, age, culture, 
gender, etc.) 

12. was available during scheduled office hours. 

13. respected the scheduled starting and stopping times of the class. 

14. graded and returned course assignments and examinations to me in a timely fashion. 

15. is someone from whom I would enjoy taking another course. 



The Course: 

16. helped me improve my communication skills. 

17. increased my understanding of the subject matter. 

18. provided experience with new or improved technical skills specific to the subject matter (e.g., laboratory techniques, 
artistic skills, clinical techniques, etc.) 

19. improved my abilities to access information beyond the textbook(s) (e.g., library, Internet, World Wide Web, data bases, 
interviews, etc.) 

20. provided me with opportunities for problem-solving, critical thinking or decision-making. 

21 . provided opportunities for me to apply the materials and information learned in this course. 

22. included activities involving a variety of methods and approaches designed to clarify the material. 

23. I am aware of cheating, plagiarism, or other forms of dishonesty occurring in the class. 

YES = a NO = b 



Instructor’s or Department's Questions: 

24. 

25. 

26. 

27. 

28. 

PLEASE TURN THIS SHEET OVER AND COMPLETE BOTH QUESTIONS ON THE BACK student: 

Please fill In the following information: 
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CATALOG NUMBER: COURSE CODE NUMBER: 

TERM & YEAR: INSTRUCTOR'S NAME: 

NARRATIVE 

INSTRUCTIONS: Please comment, using the space below, on the following topics. Your written comments will be returned to the 
instructor after the term has ended. (You may want to PRINT to protect your anonymity.) 



a. THE STRENGTHS OF THIS COURSE AND ITS TEACHER. 

(Use the space be/ow; DO NOT write your comments on the scantron sheet.) 



b. RECOMMENDATIONS FOR IMPROVEMENT. 

(Use the space below; DO NOT write your comments on the scantron sheet.) 
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